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ABSTRACT 


Thia  atudy  deals  with  decision  making  under  uncertainty,  and 
more  specifically  with  a  set  of  postulates  (based  on  the  concepts  of 
utility  and  personal  probability)  for  consistent  decision  making. 

It  explores  the  degree  to  which  these  postulates  are  good  descriptions 
of  actual  behavior  and  the  degree  to  which  they  seem  to  be  good  norms. 
Experiments  in  the  form  of  decision  problems  based  on  each  of  the 
postulates  were  given  to  experienced  business  executives.  In  addition 
to  obtaining  basic  responses  to  these  problems,  a  systematic  follow-up 
was  made  both  in  written  and  later  in  oral  form  in  order  to  probe  the 
normative  implications  of  the  postulates. 

We  conclude,  based  on  the  basic  responses,  that  the  decision 
making  theory  under  study  is  not  a  very  good  description.  However — and 
this  is  the  important  point — the  subjects  tend  to  regard  most  of  the 
deviations  from  the  theory  as  mistakes  and  if  given  the  opportunity 
will  correct  them.  This  suggests  the  importance  of  the  postulates  in 
training  decision  makers  in  both  how  to  structure  decision  problems 
and  also  how  to  police  their  own  decisions. 


1.  INTRODUCTION 


Most  decisions  of  importance  are  made  under  conditions  of 
uncertainty*  In  cl  sing  among  various  courses  of  action  we  are  not 
sure  what  the  outcome  of  any  particular  action  will  be.  A  number  of 
decision  making  theories  have  been  proposed  for  such  situations.  In 
this  paper  we  shall  deal  with  one  particular  theory  of  decision  making 
under  uncertainty.  This  theory  is  based  on  the  concepts  of  personal 
probability  and  utility  and  may  be  stated  in  the  form  of  a  set  of 
postulates.  Various  sets  of  postulates  have  been  formulated;  the 
postulates  we  shall  use  are  essentially  the  same  as  those  of  L.  J. 

Savage  [1954]. 

This  decision  theory  is  normative  rather  than  descriptive.  That 
is,  it  is  a  theory  of  how  individuals  should  make  decisions  rather  than 
how  they  do  make  decisions.  Several  challenges  to  the  normative  impli¬ 
cations  of  the  theory  have  been  made  in  the  form  of  conjectures  and 
"paradoxes."  These  have  generally  been  unsupported  by  empirical  evidence. 
On  the  other  hand,  some  researchers  have  supposed  the  theory  to  be 
descriptive  and  have  generated  much  data.  Unfortunately,  many  of  these 
studies  do  not  have  the  presumed  relevance  to  the  theory.  In  this  study 
we  attempt  to  discover  the  degree  to  which  the  postulates  are  good 
descriptions  (of  actual  behavior)  and  the  degree  to  which  they  seem  to 
be  good  norms. 
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A  descriptive  theory  can  be  Judged  by  its  explanatory  or 
predictive  ability.  It  is  more  difficult,  though,  to  Judge  a 
normative  theory.  Presumably  adopting  a  good  normative  theory  will 
lead  to  "better"  results.  But  "better"  in  what  sense?  The  criteria 
must  be  specified  and  will  often  be  part  of  the  theory  itself.  One 
condition  we  might  expect  a  good  normative  theory  to  satisfy  is  that 
it  should  seem  reasonable  to  individuals  with  expertise  in  the  domain 
of  usage.  Thus,  we  would  expect  a  good  normative  theory  of  decision 
to  seem  reasonable  to  successful,  practicing  decision  makers. 

To  this  end  we  present  some  expei imantal  results  based  on  deci¬ 
sion  problems  given  to  a  group  of  business  executives.  These  executives 
were  drawn  from  the  Executive  Development  Program  at  the  University  of 
California,  Los  Angeles.  This  program  trains  upper-middle  level  execu¬ 
tives  for  top  management  positions.  The  average*  executive  can  be 
described  as  42  years  of  age,  holding  a  college  degree,  earning  $25,000/year 
as  a  division  manager  in  a  company  of  about  6,000  employees. 

In  a  series  of  four  experimental  sessions  (of  about  one  hour  each) 
held  in  March  and  April  1964,  decision  problems  based  on  each  of  the 
postulates  were  given  to  the  subjects.  Thirty-eight  executives  became 
regular  subjects,  and  with  only  a  couple  of  minor  exceptions  completed 
all  the  material.  In  this  paper  we  can  only  briefly  summarize  the 


*  The  figures  are  based  on  median  values  for  each  characteristic 
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experiments  and  the  results.  For  completeness  we  shall,  mention  the 
results  for  each  postulate.  We  shall,  however,  particularly  emphasize 
those  experiments  and  results  having  the  greatest  relevance  to  the 
topic  of  uncertainty.  For  a  more  complete  treatment  the  reader  is 
referred  to  MacCrimmon  [1965]  and  a  series  of  forthcoming  working 
papers. 

In  order  to  probe  the  normative  implications  of  the  postulates 
we  were  not  only  concerned  with  the  subjects  *  initial  response  to  the 
decision  problems,  but  we  were  interested  in  their  behavior  when  they 
were  given  an  opportunity  to  reflect  on  their  answers—and  especially 
after  they  were  presented  with  arguments  conforming  to,  and  conflicting 
with,  the  postulates.*  For  this  reason,  the  decision  problems  were 
specially  constructed  so  that  the  strongest  possible  counter-argument 
to  each  postulate  could  be  made.  This  puts  the  postulates  to  a  very 
severe  test,  perhaps  more  severe  than  would  arise  in  most  real-world 
decision  situations. 

Although  it  is  desirable  to  present  as  much  experimental 
material  as  possible  in  a  written  form  (to  allow  replication  by  other 
researchers),  it  is  very  important  in  experiments  such  as  these  to 
have  an  opportunity  to  follow-up  on  interesting  or  confusing  written 
responses.  Thus,  after  all  the  written  experimental  material  had  been 
completed,  a  terminal  interview  (of  median  length,  30  minutes)  was 

*  These  arguments  were  presented  in  a  written  form  and  will  be  described 
in  more  detail  in  later  sections  of  this  paper. 


held  with  each  subject  individually.  These  interviews  are  an 
important  part  of  this  study,  but  because  of  constraints  on  the 
length  of  this  paper  it  is  feasible  to  present  only  a  small  part 
of  the  interview  results.  The  interested  reader  is  referred  to 
the  reports  mentioned  above. 

In  the  next  six  sections  of  this  paper,  we  shall  state  the 
appropriate  postulate,  present  one  of  the  experiments  based  (primarily) 
on  that  postulate,  describe  the  experimental  procedures,  and  discuss 
the  results  of  the  relevant  experiments.  Before  presenting  this 
material,  however,  we  need  to  introduce  the  following  basic  concepts* 

Alternatives  (also  acts  or  actions )t 

a,  b,  c  c  A  ,  the  set  of  all  possible  actions 

States: 

x  e  X  ,  the  set  of  all  possible  states  of  the  world 
Events : 

zQx 

n 

If  X  and  z  /V.0  (aU  !,J,  i/j), 

i«l  1  1  J 

we  call  {zj  an  n-fold  partition  of  X. 

zC  -  {x  t  X  :  x  i  z}  ,  and  is  called  the  complement  of 
Consequences  (also  outcomes  or  rewards): 

(a,x)  e  A  X  X 

If  (a,x)  is  a  constant  for  all  xez,  we 


write  (a,z). 
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These  concepts  plus  the  relation  of  preference  provide  the 
basle  for  the  postulates.  The  preference  relation  is  defined  as 
follows:  if  the  decision  maker  chooses  b  when  both  a  and  b 

are  available,  we  Bay  that  a  "is  not  preferred  to"  b  ,  and  write 
a  <  b.  As  a  corollary,  if  a  ^  b  and  also  b  ^  a,  we  say  that  the 
decision  maker  "is  indifferent  between"  a  and  b  ,  and  write  a^b  ; 
if  a  ^  b  but  it  is  not  true  that  b  ^  a  ,  we  say  that  b  "is  strictly 

preferred  to"  a  ,  and  write  a  <  b  .  We  later  (Postulate  4)  use  the 

preference  relation  to  develop  the  relation  "is  not  more  probable  than" 
between  events.  This  relation  is  denoted  as  <•  • 

2.  POSTULATE  1:  COMPLETE  ORDERING 

The  first  postulate  we  consider  asserts  that  all  alternatives 
are  oomparable  and  that  choices  are  transitive.  A  formal  statement 
of  the  postulate  is  given  below. 

Postulate  1:  For  every  a,  b,  c  e  A 

(1)  a  <  b  or  b  a 

(2)  if  a  <  b  and  if  b  <  c  ,  then  a  ^  c  • 

The  first  part  of  the  postulate  (comparability)  was  not  an 
explicit  part  of  our  experiments.  We  note  though  that  at  no  point 
did  a  subject  refuse  to  make  a  choice.  The  second  part  of  the 
postulate  (transitivity)  was  the  basis  for  a  particular  set  of 
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experiments.  These  experiments  will  be  treated  only  briefly  in  this 
paper  because  although  transitivity  is  a  very  important  concept, 
it  is  not  a  notion  especially  concerned  with  the  topic  of  uncertainty. 

As  an  example  of  the  experiments  used,  the  context  of  the 
first  experiment  was  as  follows : 

Assume  you  are  President  of  &  company  that  makes 
small  appliances.  Suppose  your  company  has  extensively 
redesigned  and  improved  one  item  in  your  product  line. 

You  must  now  make  the  final  decision  on  the  new  price. 

You  feel  that  the  important  criteria  in  evaluating 
a  pricing  policy  are  expected  return  and  expected  share 
of  market.  You  have  directed  your  market  research  staff 
to  prepare  these  figures  for  the  various  possible  pricing 
policies.  You  have  every  reason  to  believe  in  the 
reliability  of  the  figures. 

The  subject  was  asked : 

Which  of  the  following  pricing  policies,  A  or  B, 
do  you  prefer?  Please  circle  your  choice. 

A)  Expected  return;  10^ 

Expected  share  of  market:  kO% 

B)  Expected  return:  20?> 

Expected  share  of  market:  7'0% 


A  total  of  4  distinot  alternatives  were  used  (the  others  being 

C)  Expected  return:  5%,  Expected  share  of  market:  5 0 %»  and  D)  Expected 
return:  15%,  Expected  share  of  market:  30%),  All  6  binary  combinations 
of  these  4  alternatives  were  presented  (interleaved  with  some  other 
transitivity  experiments).  This  structure  means  that  the  subject  had 
4  chances  to  make  2  intransitivities  on  this  experiment.  [May,  1954] 
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Aftar  baing  presented  these  binary  ohoioes  the  subject  was 
asked  to  rank  the  four  alternatives.  Thus  in  addition  to  observing 
intransitivities  (i.e.,  circular  triples)  we  can  compare  his  binary 
choices  with  his  ranking  of  the  four  alternatives.  A  discrepancy 
between  the  binary  ohoice  and  the  ranking  will  be  called  a  "choice 
instability".* 

In  the  experiment  described,  only  2  of  the  38  subjects  had  an 
intransitivity  while  18  of  the  subjects  had  "choice  instabilities". 

In  the  other  two  transitivity  experiments  (with  a  similar  structure 
to  the  one  described)  one  subjeot  in  the  second  experiment  and  5 
subjects  in  the  third  had  intransitivities,  while  15  and  11  subjects, 
respectively,  had  "choice  instabilities"  in  these  experiments.  No 
subject  had  an  intransitivity  in  more  than  one  experiment.  Thus,  of 
the  114  subject-experiments  (38  x  3)>  8  subjects  had  intransitivities. 

During  the  interview,  6  of  these  8  subjects  quickly  acknowledged 
that  they  had  made  a  "mistake"  in  their  choices  and  expressed  a  desire 
to  change  their  choices.  The  remaining  two  subjects  persisted  in  the 
intransitivity  asserting  that  they  saw  no  need  to  change  their  choioes 
because  they  focused  on  different  dimensions  for  different  choice  pairs. 
The  fallacy  of  this  reasoning  was  not  apparent  to  them  in  a  five-minute 
discussion. 


*  An  intransitivity  implies  a  "choice  instability",  but  the  converse  is 
not  true. 
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Only  8  of  the  38  subjects  had  no  "choice  instabilities"  in 
the  three  experiments.  The  other  30  subjecte  (except  for  the  2  that 
persisted  in  intransitivities)  all  wished  to  change  their  choice 
(most  would  change  the  binary  choice,  but  the  difference  was  not 
significant).  They  generally  attributed  the  "choice  instability"  bo 
carelessness  in  their  reading  or  thinking. 

3.  POSTULATE  2:  (a)  THE  IRRELEVANCE  OF  IDENTICAL  OUTCOMES 

Two  implications  of  Postulate  2  will  be  discussed  in  this 
paper.  The  first,  which  maybe  called  "the  irrelevance  of  identical 
outcomes"  is  the  subject  of  this  section,  and  the  second  implication 
will  be  covered  in  the  next  section  (4).  Postulate  2  is  formally 
stated  below. 


POSTULATE  2:  If  (a,x)  ■  (a',x)  and  (b,x)  »  (b',x) 
for  every  x  c  z  ,  and  if  (a,x)  -  (b,x)  and  (a',x)  -  (b',x) 
for  every  x  e  z  ,  then  a  b  if,  and  only  if,  a'  b  *  • 

Three  experiments  concerning  the  "irrelevance"  implication 
of  Postulate  2  were  given  to  the  subjects.  The  first  experiment 
read  as  follows. * 

Assume  you  are  the  President  of  a  medium-sized  company, 
and  you  have  to  choose  among  alternative  investments  in  a 
production  process  involving  uncertain  outcomes.  Suppose 


*  Slight  changes  have  been  made  in  notation  and  format,  so  the  current 
reader  can  more  easily  relate  the  experiment  and  Postulate  2. 


further  that  the  uncertainty  can  be  related  to  drawing  a  ball 
out  of  an  urn  containing  100  balle  numbered  from  1  to  100. 

The  ball  drawn  will  determine  the  outcome. 

Which  of  the  following  two  investments  do  you  chcoae?  (Please  give 
reasons  for  your  choice.) 


Investment  a::  Invest  in  a  production  process  such 
that  if  ball  numbered  1-10  is  drawn,  it  corresponds  to 
getting  a  return  of  500^;  if  ball  number  11  is  drawn,  it 
corresponds  to  failure  of  the  process  and  your  company  will 
go  bankrupt;  and  finally  if  ball  numbered  12-100  is  drawn, 
it  corresponds  to  getting  a  return  of  5%, 

Investment  bt  Invest  in  a  production  process  such  that 
no  matter  which  ball  (numbered  1-100)  is  drawn,  it  corresponds 
to  getting  a  return  of  5%* 

After  making  a  choice  between  a  and  b,  the  subject  was  then  asked: 

Making  the  same  assumptions  as  above,  if  instead  of  investments 
a  and  b  you  are  faced  with  investments  a'  and  b',  which  would  you  choose? 
(Please  give  reasons  for  your  choice.) 


Investment  a 1 :  Invest  in  a  production  process  such 
that  if  ball  numbered  1-10  is  drawn,  it  corresponds  to 
getting  a  return  of  50C#j  however,  if  ball  numbered  11-100 
is  drawn,  your  company  will  go  bankrupt. 

Investment  b ' :  Invest  in  a  production  process,  su'  . 
that  if  ball  numbered  1-11  is  drawn,  it  corresponds  to 
getting  a  return  of  5%;  however,  if  ball  numbered  12-100 
is  drawn,  your  company  will  go  bankrupt. 


For  the  reader's  convenience  this  experiment  is  shown  in  the 


payoff  matrix  below. 
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Balls  1-10 

Ball  11 

Balls  12-100 

Investment  a 

500^  return 

bankruptcy 

5#  return 

Investment  b 

5)6  return 

5#  return 

5)6  return 

Investment  a’ 

500)6  return 

bankruptcy 

bankruptcy 

Investment  b' 

5)6  return 

5)6  return 

bankruptcy 

Postulate  2  asserts  that  a  choice  of  b  over  a  should  lead 
to  a  choice  of  b'  over  a'  ,  and  conversely.  (Note  thnt  event  z 
corresponds  to  Balls  1-11,  and  event  z  corresponds  to  Balls  12-100. ) 

This  type  of  problem  has  been  suggested  by  M.  Allais  [1953, 
1953a]  and  G.  Morlat  [1953].  They  conjecture  that  individuals  when 
confronted  with  such  situations  will  choose  alternative  b  over  a  , 
and  alternative  a1  over  b'  .  The  assertion  is  made  [Allais,  1953] 
that  even  upon  reflection  individuals  will  maintain  such  a  choioe. 
Persistence  in  Buch  a  choice  (or  in  a  and  b')  is,  of  course,  in 
conflict  with  Postulate  2. 

In  our  experiments,  after  answering  the  above  questions  the 
subject  was  presented  with  two  prepared  answers  to  the  same  problem. 

He  was  told  both  answers  had  been  given  by  executives  in  a  previous 
session.  He  was  asked  to  provide  a  critique  of  each  answer  and  to 
select  the  one  that  was  more  logical. 
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One  prepared  answer  conformed  with  Postulate  2.  It  chose 
alternatives  b  and  b*  and  gave  reasons  to  the  effect  that  the 
outcomes  for  the  pair  a  and  b  ,  and  for  the  pair  a1  and  b*  , 
were  identical  if  a  ball  numbered  12  through  100  was  drawn,  and,  on 
the  basis  of  taste,  b  was  preferred  to  a  when  considering  balls 
1-11,  but  since  b1  is  identical  to  b  ,  and  a1  is  identical  to 
a  ,  this  Implied  b*  should  then  be  chosen  over  a'  • 

The  other  prepared  answer  had  choices  that  conflicted  with 
Postulate  2.  The  reasons  presented  were  similar  to  those  suggested 
by  Allaie  [19533*  The  choices  made  were  b  and  a'  .  The  Justification 
for  b  was  that  it  was  safer — i.e.,  why  take  a  chance  on  bankruptcy — 
whereas  the  Justification  for  a'  was  that  both  a*  and  b'  involved 
a  high  (and  almost  indistinguishable)  chance  of  bankruptcy  and  so  one 
might  as  well  aspire  to  a  500$  return. 

In  the  decision  problem  described  above  (the  first  of  three  quite 
similar  experiments),  slightly  over  60JC  of  the  (36  participating)  subjects 
gave  initial  answers  consistent  with  Postulate  2.  Thus,  almost  40/6  had 
answers  in  conflict  with  Postulate  2.  These  same  percentages  of  conformity- 
conflict  responses  were  found  in  the  other  two  similar  experiments.  (The 
second  experiment  changed  some  of  the  material  to  a  qualitative  form, 
and  the  third  experiment  reversed  the  ball  numbers  and  narrowed  the  quan¬ 
titative  outcomes.)  A  high  degree  of  apparent  conflict  with  Postulate  2 
thus  exists  in  the  actual  responses  of  the  subjects. 
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Information  on  the  extent  to  which  the  figures  represent 
accidental  conformity  or  accidental  violation  may  be  obtained  from 
the  (written)  responses  to  the  prepared  answers.  On  the  first 
experiment,  only  about  2(#  of  the  subjects  selected  the  conforming 
answer,  while  about  8C^  selected  the  conflicting  answer  as  being  the 
more  logical  one.  Note  then  that  many  individuals  who  themselves 
gave  conforming  answers  disagreed  with  the  prepared  conforming  answer. 
(It  is  also  true  that  some  subjects  initially  giving  violating  answers 
disagreed  with  the  prepared  violating  answer. )  These  percentages 
shifted  toward  conformity  on  the  second  experiment,  with  about  50^ 
of  the  subjects  selecting  the  conforming  answer  and  about  50/f  of  the 
subjects  selecting  the  violating  answer.  (There  was  no  presentation 
of  prepared  answers  in  the  third  experiment. ) 

Some  logical  lapses  on  the  part  of  the  subjects  are  thus 
apparent.  Even  though  an  individual  himself  may  conform  or  violate 
Postulate  2,  he  apparently  cannot  recognize  a  complementary  conforming 
or  violating  answer.  Perhaps  the  prime  reason  for  this  response 
pattern  was  uncovered  during  the  interview.  The  subjects  had  great 
difficulty  separating  logical  deductions  from  a  set  of  premises 
(the  operation  that  was  requested  here)  from  the  beliefs  underlying 
the  premises  themselves.  Thus,  for  example,  if  an  individual  selected 
a  and  a*  (a  conforming  answer)  he  may  select  the  violating  prepared 
answer  of  b  and  a*  because  the  prepared  conforming  answer,  b  and 
b’  ,  is  composed  of  two  completely  different  actions.  Reasons  given 
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for  this  choice  would  often  be  that  the  (b,b')  prepared  answer  was 
"too  conservative,"  rather  than  asserting  it  was  less  logical  than 
the  other  prepared  answer.  (Analogous  responses  were  given  in  the 
other  cases.)  Reasons  given  for  the  prepared  answer  choice  often  had 
no  connection  with  the  subject's  own  answers  and  reasons  given  just  a 
few  minutes  earlier. 

The  oral  interview  provided  an  opportunity  to  clear  up  some  of 
this  confusion  between  beliefs  and  logical  deductions.  After  such  a 
discussion  most  of  the  subjects  tended  to  move  toward  conformity  with 
the  postulate-based  reasoning.  This  trend  can  even  be  noted  in  the 
strictly  written  part  of  the  experiment  by  comparing  the  figures  given 
above  for  the  responses  to  the  prepared  answers  in  the  first  and  second 
experiments.  The  shift  became  more  pronounced  during  the  interview 
with  about  75%  of  the  subjects  indicating  their  complete  agreement  with 
the  postulate-based  reasoning.  Another  11%  indicated  agreement  for  all 
instances  except  where  one  alternative  has  a  sure  outcome  (for  example,  in 
the  experiment  described  earlier,  alternative  b  always  gives  a  5%  return). 

Approximately  14%  of  the  subjects  indicated  their  continued  dis¬ 
agreement  with  the  applicability  of  the  postulate-based  reasoning  to 
such  decision  problems.  The  reason  most  often  given  by  these  subjects 
was  that  the  problem  could  not  be  decomposed  as  the  postulate-based 
reason  implied.  Comments  such  as  "you  can't  disregard  what  happens  if 
balls  12-100  are  drawn,"  "you  can't  ignore  such  a  big  part  of  the 


*  —  • 
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problem,  "  and  "you're  only  looking  at  part  of  the  picture,"  were 
typical  of  this  group.  Attempts  to  explain  that  events  were  not 
being  "ignored"  were  unsuccessful.  It  should  be  observed  though 
that  all  these  subjects  would  be  willing  to  conform  to  Postulate  2 
if  tho  event  having  identical  outcomes  was  not  so  probable.  Thus, 
that  we  get  persistent  violation  of  Postulate  2  is  primarily  due  to 
the  very  special  nature  of  these  constructed  situations. 

4.  POSTULATE  2:  (b)  RISK  versus  UNCERTAINTY 

In  this  section  we  shall  consider  another  implication  of 
Postulate  2  (formally  stated  on  p.  8).  This  interpretation  may  be 
called  "risk  versus  uncertainty."  We  can  develop  the  meaning  of  this 
most  usefully  in  the  context  of  one  of  the  three  experiments  actually 
given  to  the  subjects.  The  core  of  the  first  experiment  read  as 
follows : 

1,  Which  of  the  following  wagers  do  you  prefer? 

(a)  $1000  if  the  top  card  in  an  ordinary,  well-shuffled 
deck  of  playing  cards  is  red;  $0  if  black. 

(b)  $1000  if  the  closing  price  tomorrow  of  Pierce  Industries 
on  the  American  Stock  Exchange  is  higher  than  the  closing 
price  today;  $0  if  not  higher. 

(c)  Either  (a)  or  (b) 


Please  give  reasons  for  your  choice. 
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2.  Which  of  the  following  wagers  do  you  prefer? 

(a)  $1000  if  the  top  card  in  an  ordinary,  well-shuffled 
deck  of  playing  cards  is  black;  $0  if  red. 

(b)  $1000  if  the  closing  price  tomorrow  of  Pierce  Industries 
on  the  American  Stock  Exchange  Is  not  higher  than  the 
closing  price  today;  $0  if  higher. 

(c)  Either  (a)  or  (b). 

Please  give  reasons  for  your  choice. 


This  abbreviated  form  of  the  first  experiment  can  serve  to  demonstrate 

the  main  focus  of  our  interest  in  this  section,  that  is,  a  "risk/uncertainty" 

implication  of  Postulate  2. 

The  applicability  of  Postulate  2  to  this  experiment  can  perhaps  best 
be  seen  by  considering  the  payoff  matrix  representation  below. 


-^Events 
Actions" — 

Stock  higher 
& 

Card  black 

Stock  not  higher 
& 

Card  red 

Stock  higher 
& 

Card  red 

Stock  not  higher 
& 

Card  black 

1(a) 

$0 

$1000 

$1000 

$0 

1(b) 

$1000 

$0 

$1000 

$0 

2(b) 

$0 

$1000 

$0 

$1000 

2(a) 

$1000 

$0 

$0 

$1000 

Now  making  a  correspondence  with  the  notation  of  Postulate  2  (p.  8), 
let  us  associate  the  (joint)  events  "stock  higher  &  card  black"  and  "stock 
not  higher  &  card  red"  with  z  ,  the  (joint)  events  "stock  higher  &  card 
red"  and  "stock  not  higher  &  card  black"  with  zc  .  Let  us  further 
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associate  the  actions  1(a)  with  a  ,  1(b)  with  b  ,  2(b)  vdth  a'  ,  and 

2(a)  with  b'  .  For  convenience  in  the  following  discussion  let  us 

refer  to  1(a)  as  a  "bet"  on  the  card  being  red,  1(b)  as  a  "bet"  on  the  stock 

price  being  higher,  2(b)  as  a  "bet1,  on  the  stock  price  not  being  higher, 

/  \  * 

and  2(a)  as  a  "bet"  on  the  card  being  black. 

Using  these  correspondences,  Postulate  2  asserts  that  if  you 
prefer  a  bet  on  the  card  being  red  over  a  bet  on  the  stock  price  being 
higher,  you  then  should  prefer  a  bet  on  the  stock  price  not  being  higher, 
than  a  bet  on  the  card  being  black.  Note  that  the  event  "black"  is  the 
complement  of  the  event  "red",  ar  d  the  event  "not  higher"  is  the  comple¬ 
ment  of  the  event  "higher".  Postulate  2  thus  asserts  that  if  you  prefer 
a  bet  on  one  event  over  a  bet  on  a  second  event,  then  you  should  prefer 
a  bet  on  the  complement  of  the  second  event  over  a  bet  on  the  complement 
of  the  first  event  (given  the  same  reward  conditions).  This  relationship 
holds  regardless  of  the  nature  of  the  events.  Thus,  within  the  context 
of  this  theory  it  is  not  meaningful  to  distinguish  between  the  nature  of 
events . 

A  notion  that  is  in  conflict  with  this  is  the  dichotomy  between 
"risky  events"  and  "uncertain  events"  that  is  sometimes  made.  Under  such 
a  view,  the  card  events  would  be  called  "risky"  because  most  people  would 
be  willing  to  assign  a  (common)  relative  frequency  to  them,  while  the 
stock  events  would  be  called  "uncertain"  because  most  people  would  not 


*  By  a  bet  on  an  event  we  mean  an  alternative  such  that  the  decision  maker 
gains  more  if  that  event  occurs  than  he  does  if  it  does  not  occur. 
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assign  (common)  relative  frequencies.  This  leads  to  an  incomparability 
between  “risky  events"  and  "uncertain  events"  and  to  a  different  pro¬ 
cedure  for  dealing  with  them.  Thus,  with  such  a  view,  it  would  not  be 
inconsistent  to  prefer  the  bet  on  "red1'  to  the  bet  on  "higher"  and  then 
to  prefer  the  bet  on  "black"  to  the  bet  on  "not  higher". 

Ibis  conclusion,  and  the  type  of  experiment  it  a  el  r ,  has  been  sug¬ 
gested  by  D.  Ellsberg  [1961]  as  a  situation  in  which  people  do  not  act  in 
a  manner  consistent  with  Postulate  2.  He  contends  that  on  the  basis  of 
presenting  similar,  but  informal,  problems  to  some  top  decision  theorists, 
these  individuals  generally  do  choose  actions  1(a)  and  2(a),  i.e.,  the 
bets  on  the  "risky  events"  over  the  bets  on  the  "uncertain  events". 
Ellsberg  further  claims  that  even  upon  reflection  individuals  will  tend 
to  persist  in  these  choices. 

In  our  experiments  the  subject  was  asked  to  make  choices  and  give 
reasons  in  each  of  the  six  possible  pairs  of  wagers  (i.e.,  bets  on  two 
events  and  their  complements  taken  pairwise)  with  the  (identical)  reward 
conditions  used  in  the  example  at  the  beginning  of  this  section.  However, 
it  should  be  noted  that  a  choice  of  a  bet  on  "red"  may  not  indicate  strict 
preference  but  only  indifference.  Thus,  a  choice  of  a  bet  on  "red"  and, 
in  a  later  pair,  a  choice  of  a  bet  on  "black"  may  not  be  inconsistent 
with  Postulate  2  if  they  merely  indicate  indifference.  Therefore,  a  second 
set  of  reward  conditions  was  also  used.  Under  these  conditions  the  stock 
bets  still  paid  off  $1000  if  correct,  but  the  payoff  on  the  card  bets 


-i8 


was  reduced  to  $990.  It  is  easily  seen  that  a  choice  of  both  the  "red" 
and  "black"  bets  under  these  conditions  is  inconsistent  Kith  Postulate  2. 
Altogether,  then,  twelve  pairs  of  wagers  were  presented  to  the  subject 
in  a  single  experiment. 

Three  "risk/uncertainty"  type  experiments  were  given  to  the  sub¬ 
jects.  The  first  has  been  discussed  as  the  example  in  thi  s  section.  The 
third  was  very  similar,  the  only  change  being  a  replacement  of  the  card 
events  ("red"  or  "black")  by  coin  events  ("head"  or  "tail")  and  a  replace¬ 
ment  of  the  stock  events  ("higher"  or  "not  higher")  >y  GNP  events 
("U.S.  '64  GNP  lees  than  620  billion"  or  "U.S.  ‘64  GNP  at  least  620 
billion").  The  second  experiment  involved  investments  in  two  foreign 
countries  with  historical  frequencies  being  given  for  one  country  (a 
"risky"  case)  while  no  probabilities  were  assigned  for  the  other  country 
(a  case  of  "uncertainty"). 

In  addition  to  asking  for  the  subject's  own  responses  on  the  first 
experiment,  two  different  sets  of  choices  and  reasons  were  presented  (in 
written  form)  to  him  for  his  critique.  One  set  was  consistent  with 
Postulate  2,  using  informal  probability  type  arguments,  while  the  other 
was  inconsistent  with  Postulate  2  in  that  it  gave  "risk/uncertainty"  type 
arguments— stating  that  the  card  odds  were  known  whereas  the  stock  odds 
were  not  known.  Both  sets  of  written  reasons  were  entirely  verbal  in 


form 
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Thirty- five  of  the  38  subjects  completed  the  three  experiments 
and  the  corresponding  part  of  the  terminal  interview.  Various  kinds  of 
answers  inconsistent  with  Postulate  2  can  be  identified  [MacCrimmon,  1965], 
but  here  we  shall  only  be  concerned  with  the  "risk/uncertainty"  ones. 

(Most  of  the  others  were  later  labeled  as  "mistakes"  by  the  subjects.) 

In  the  first  experiment  (the  example  at  the  beginning  of  this 
section)  only  3  subjects  had  "risk/uncertainty"  violations  of  Postulate  2, 
in  the  third  experiment  only  5  subjects  had  such  violations,  while  in 
the  second  experiment  19  subjects  had  "risk/uncertainty"  violations.  Later 
questioning  of  the  subjects  indicated  that  while  the  second  experiment  pro¬ 
vided  a  less  artificial  decision  context  it  afforded  an  opportunity  to 
make  many  irrelevant  and  changing  assumptions  that  led  to  an  unusually 
high  degree  of  conflict  with  Postulate  2. 

Considering  all  three  experiments  (even  though  the  second  has 
the  defects  mentioned  above),  no  subjects  had  three  "risk/uncertainty" 
violations,  only  3  subjects  had  two  "risk/uncertainty"  violations,  and 
11  subjects  had  no  "risk/uncertainty"  violations.  The  remaining  21 
subjects  had  one  "risk/uncertainty"  Violation,  with  17  of  these  occurring 
on  the  second  experiment.  If  we  separately  consider  only  the  first  and 
third  experiments  (the  two  most  closely  corresponding  to  Ellsberg's), 
we  find  that  28  subjects  had  no  "risk/uncertainty"  violations,  6  subjects 
had  one,  while  only  1  subject  had  a  "risk/uncertainty"  violation  in 
both  experiments. 
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In  the  first  experiment  a  relatively  higher  proportion  agreed 
with  the  violating  answer  than  gave  violating  answers  themselves; 
however,  a  majority  still  thought  that  the  conforming  answer  was  more 
logical.  Eighteen  subjects  selected  the  conforming  answer,  11  subjects 
selected  the  "risk/uncertainty"  violating  answer,  while  the  other  6 
subjects  thought  neither  was  logical.  Contrary  to  the  responses  to 
the  prepared  answers  for  the  experiments  discussed  in  the  previous 
section  (section  3),  there  was  an  insignificant  amount  of  "cross-over"  (i.e., 
subjects  giving  one  response  themselves,  then  later  selecting  the  other). 

At  the  terminal  interview,  the  reasons  for  the  subject's  particular 
choices  were  pursued  with  emphasis  being  placed  on  those  seemingly  dis¬ 
tinguishing  between  "risk"  and  "unce-tainty".  The  only  subject  giving 
"risk/uncertainty"  choices  in  his  written  answers  to  the  first  and 
third  experiments  persisted  in  this  distinction  during  the  interview. 

He  insisted  that  he  knew  the  odds  on  card  and  coin  bets  but  not  on 
bets  involving  stock  price  increases  or  the  level  of  GNP;  and  further 
he  stated  that  he  would  select  bets  in  which  he  knew  the  odds — even 
at  a  slight  penalty  (in  terms  of  the  payoffs).  Of  the  other  6  subjects 
having  (single)  "risk/uncertainty"  violations  in  either  the  first  and 
third  experiment,  all  but  one  (with  a  second  subject  unsure)  concluded 
that  such  a  distinction  was  not  reasonable,  and  they  indicated  a  wish  to 
change  their  choices.  However,  in  the  second  experiment,  slightly 
over  half  (10  of  19  subjects)  did  not  wish  to  change  a  "risk/uncertainty" 
violation.  In  general,  this  may  be  related  to  ambiguity  in  the  problem 
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stateasnt,  clarification  of  which  the  subjects  would  not  accept.  Of 
ths  other  subjects — those  initially  giving  non  "risk/uncertainty" 
violating  answers ,  almost  all  attributed  their  answers  to  "mistakes" 
and  agreed  completely  with  the  applicability  of  Postulate  2. 

In  general,  then,  subjects  do  not  persist  in  making  distinctions 
between  "risky"  and  "uncertain"  events  whan  given  an  opportunity  to 
reflect  on  their  answers — and  this  in  an  experiment  especially  designed 
to  elicit  such  violations!  Persistent  violators  are  probably  rare,  but 
one  can  agree  with  Ellsberg  [1963]  that  they  are  a  particularly  interest 
ing  group  to  study  further. 

5.  POSTULATE  3:  ADMISSIBILITY 

# 

Postulate  3  introduces  a  very  important  concept,  admissibility 
(also  called  "dominance").  This  notion  has  general  interest  beyond  a 
particular  theory  of  decision  such  as  that  considered  here.  This 
postulate  is  formally  stated  as  follows: 

POSTULATE  3:  If  is  an  n-fold  partition  of  z  , 

and  if  (a,  z^)  (b,  z^)  for  all  z^  c  jz^  , 

i  ■  1,  2, . . . ,  n  ,  then  a  b  given  z  •  If,  in 

addition,  (a,  z^  )  <  (b,  z^  )  for  some  non-null 
o  o 

*i  e  lzil  *  ^hen  a  ^ b  S^en  z  . 


*  Corresponding  to  Savage's  Theorem  3  [1954,  p.  26 3. 
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When  the  above  condi tione  hold  wo  say  that  alternative  a  ie 
"inadmis Bible",  or  that  alternative  b  "dominates"  alternative  a  • 

The  conoept  of  admissibility  appeals  very  strongly  to  one's 
common  sense.  It  also  can  have  great  practical  significance  in 
decision  problems.  It  gives  us  a  way  of  choosing  among  complex  actions 
in  particular  situations  after  having  established  a  preference  among 
consequences. 

As  part  of  the  experiments  on  the  independence  of  beliefs  and 
tastes  (to  be  discussed  in  the  next  section),  material  dealing  with 
admissibility  was  included.  In  these  experiments  the  subjects  were 
asked  to  rank  a  number  of  alternative  wagers  from  the  one  they  pre¬ 
ferred  most  to  the  one  they  least  preferred.  Of  the  six  pages  of 
wagers  only  the  last  three  are  of  relevance  here;  they  each  contained 
12  wagers. 

Three  pairs  of  wagers  form  the  te3t  of  admissibility. 

a, :  Win  $1200  if  the  Gross  National  Product  for  1964 
is  over  $610  billion;  lose  $400  otherwise, 
b  :  Win  $1200  if  the  Gross  National  Product  for  1964 
is  over  $600  billion;  lose  $400  otherwise. 

&  •.  Win  $1200  if  Goldwater  receives  the  Republican 

nomination  for  President  in  1964;  lose  $400  otherwise. 
b2*  Win  $1200  if  Goldwater  or  Rockefeller  receives  the 
Republican  nomination  for  President  in  1964;  lose 
$400  otherwise. 

a~:  Win  $1200  if  the  price  of  Syntax  on  the  American  Stock 
*  Exchange  is  at  least  twice  its  current  price  by  the 
end  of  this  year;  lose  $400  otherwise. 
b~:  Win  $1200  if  the  price  of  Syntax  on  the  American  Stock 
J  Exchange  is  higher  than  its  current  price  by  the  end 
of  this  year;  lose  $400  otherwise. 
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On  each  of  the  three  pages,  two  of  these  pairs  appeared;  thus,  each  pair 
was  presented  twice.  Different  reward  conditions  were  used  on  each  page. 

The  applicability  of  the  postulate  of  admissibility  to  the  above 

pairs  should  be  quite  apparent.  In  each  case  the  b  alternative  dominates 

* 

the  a  alternative  (i.e.,  a^  ^  b^  for  i  ■  1,  2,  3)  •  Ranking  the  a 
alternative  higher  than  the  b  alternative  is,  thus,  a  violation  of  the 
admissibility  postulate. 

Thirty-seven  of  the  38  regular  subjects  completed  the  admissibility 
experiment  and  participated  in  the  interview.  Of  the  37  subjects,  12  had 
at  least  one  inadmissibility,  and  5  of  the  12  had  multiple  inadmissibilities. 
Four  of  the  5  had  two  inadmissible  choices,  while  the  other  subject  had 
four  inadmissib.  Titles.  (The  maximum  number  of  inadmissibilities  possible, 
of  course,  was  six,  that  is,  three  pairs  of  wagers  presented  twice.) 

Because  convincing  written  counter-arguments  to  the  concept  of 
admissibility  could  not  be  constructed,  the  subjects  were  not  forced  to 
reflect  on  their  answers  until  the  interview.  After  giving  the  subjects 
such  an  opportunity  during  the  interview,  almost  all  subjects  with  inad¬ 
missibilities  quickly  indicated  that  they  would  revise  them.  They  made 
comments  such  as,  "That's  just  a  mistake",  "I  misread  it",  and  "That 
other  one  is  obviously  better". 

Two  subjects  did  not  recognize  the  Inadmissibility  immediately  and 
they  were  questioned  further  about  their  choices  (e.g.,  questions  such  as 
"Do  you  know  what  happens  if  GNP  is  between  600  and  610?").  These  sub¬ 
jects  then  perceived  the  inadmissibility.  One  subject  stated  that  "It 


*  Assuming  a  larger  money  amovnt  is  preferred  to  a  smaller  amount. 


seems  obvious  now,  .  ,  .  but  I  didn’t  look  st  it  that  way".  The  other 
subject  said,  "I  don't  know  what  I  could  have  been  thinking  of". 

Thus  no  subject  persisted  in  an  inadmissible  choice.  Those 
subjects  initially  giving  inadmissible  choices  generally  attributed 
them  to  carelessness  in  their  reading  or  thinking  about  the  problem. 

6.  POSTULATE  hi  INDEPENDENCE  OF  BELIEFS  AND  TASTES 

The  postulate  we  deal  with  in  this  section  asserts  that  the 
decision  maker's  beliefs  should  be  independent  of  his  tastes.  He 
should  be  subject  neither  to  wishful  thinking  nor  to  persecution 
mania.  Postulate  4  may  be  formally  stated  as  follows. 

POSTULATE  4:  For  every  a,  b  c  A  and  z^,  z^C,  z2,  z^ciX, 
if  (a,  Zj0)  ~(b,  z2C)  <  (a,  z^^b,  z2),  then 
z^  <•  z2  if,  and  only  if,  a  ^  b  . 

The  alternatives  a  and  b  may  be  thought  of  as  "bets"  on  events 
z^  and  z2  ,  respectively,  in  the  same  sense  as  the  term  was  used  in 
section  4. 

Subjects  were  asked  to  rank,  from  most  preferred  to  least  pre¬ 
ferred,  all  bets  on  a  page  containing  either  8  or  12  bets.  There  were  6 
such  pages.  The  bets  were  grouped  into  quadruples  and  various  combinations 
of  quadruples  were  presented  in  a  random  order  on  a  page.  Each  bet  on  the 
same  page  had  identical  reward  conditions,  but  each  page  had  a  different  set 
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of  rewards.  The  reward  conditions  comprised  all  possible  combinations 
of  losses  and  gains,  and  positive  and  negative  expected  money  values. 

An  example  of  the  type  of  quadruple  and  reward  condition  used 
is  given  below. 

(a)  Win  $1200  if  a  fair  coin  falls  head  up;  lose  $k00 

otherwise. 

(b)  Win  $1200  if  the  gold  outflow  exceeds  the  gold  inflow 

in  the  U.S.  this  year;  lose  $400  otherwise. 

(c )  Win  $1200  if  Britain  enters  the  Common  Market  this  year; 

lose  $400  otherwise. 

(d)  Win  $1200  if  the  price  of  the  stock  Syntex  on  the  American 

Stock  Exchange  at  the  end  of  this  year  will  be  at  least 
twice  its  current  price;  lose  $400  otherwise. 


A  total  of  four  such  quadruples  were  used. 

Postulate  4  asserts  that  these  bets  should  be  ranked  in  the 
same  order  irrespective  of  the  particular  reward  conditions  (so  long 
as  the  first  is  preferred  to  the  second — for  example,  when  win  $1200, 
lose  $400  are  replaced  by  win  $0,  lose  $300,  respectively).  Thus, 
on  each  page  that  a  quadruple  appears  it  should  be  ranked  in  identical 
order.  The  bets  in  a  given  quadruple  always  appeared  mixed  in  with 
those  of  one  or  two  other  quadruples  (the  quadruples  combined  differed 
from  page  to  page).  Subjects  were  not  allowed  to  look  back  or  to  keep 
notes. 


Thirty-seven  subjects  completed  this  experiment.  Only  two 
subjects  had  as  many  as  2  of  the  4  quadruples  ranked  identically  over 
the  3  or  4  times  the  quadruple  was  presented.  Sixteen  subjects  had 
none  of  the  4  quadruples  ranked  identically.  This  is  an  unexpectedly 
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high  degree  of  apparent  disagreement  with  the  independence  of  tastes  and 
beliefs  postulate.  That  is,  apparently  most  subjects  do  let  their  tastes 
influence  their  beliefs. 

However,  during  the  interview  there  was  a  unanimous  swing  toward 
identical  rankings  (by  each  subject  for  the  quadruples  presented).  None 
of  the  37  subjects  wished  to  maintain  even  one  different  ranking — some 
even  said  specifically  that  "the  probabilities  don't  change”.  The  viola¬ 
tion  was  primarily  attributed  to  mistakes  they  had  made  and  the  complexity 
of  the  experiment.  As  one  executive  said,  "I  mairly  consider  at  most  3 
or  4  alternatives  at  a  time,  not  8  or  12 1"  There  was  complete  agreement 
(after  the  interview)  on  the  reasonableness  of  the  implications  of  Postu¬ 
late  4. 

7.  POSTULATE  5:  EQUIVALENT  N-FOLD  PARTITION 

* 

The  fifth  postulate  is  primarily  a  structural  one.  It  asserts 
that  the  decision  maker  should  be  able  to  construct  an  n-fold  equivalent 
partition  of  the  universal  state.  The  meaning  of  this  is  implicitly 
defined  In  the  following  formal  statement  of  the  postulate. 

POSTULATE  5  s  For  any  n  ,  there  exists  an  n-fold  partition 
z  ■  {*  ,  z2,...,  zn}  of  X  such  that  for  some  a^,  a2,...,  aR  e  A  , 
1)  (ai,  ziC)  -  s  <t  -  (a^,  zA)  and  2)  — aR  . 


*  Corresponding  to  Savage's  Postulate  6. 
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In  the  experiments  we  deal  with  only  a  apecial  case  of  Postulate  5, 
the  case  where  n  ■  2  •  Thus,  we  are  examining  only  the  subjects'  ability 
to  make  an  equivalent  binary  partition.  Consider  the  following  aeries  of 
wagers . 

Please  indicate  your  choice  among  the  following  sets  of  wagers: 

(a)  Win  $1000  if  the  Dow  Jones  industrial  average  (of 

stocks  on  the  New  York  Stock  Exchange)  at  the  end 
of  this  year  is  at  least  x  ;  win  $0  otherwise. 

(b)  Win  $1000  if  the  Dow  Jones  industrial  average  at  the 

end  of  this  year  is  less  than  x  ;  win  $0  otherwise. 

(c)  Neither  or  either  (please  explain). 

The  variable  x  takes  on  the  values  600,  1000,  800,  700,  900,  650,  950, 
750,  and  850  in  that  order.  The  value  of  the  Dow  Jones  industrial  average 
at  the  time  of  the  experiments  was  around  800. 

Consistent  responses  would  select  the  "at  least"  bet  for  x  less 
than  some  value,  x  ,  and  would  switch  to  the  "less  than"  bet  for  x 
greater  than  xq  .  This  experiment  roughly  indicates  a  zeroing-in  pro¬ 
cedure  for  obtaining  equivalent  partitions;  however,  note  that  we  deal 
only  with  the  special  case,  n  ■  2  ,  and  only  coarse  intervals  are  used. 

Of  the  37  subjects  who  completed  this  experiment,  32  gave 
choices  (of  the  above  form)  from  which  a  unique,  equivalent  2-fold  parti¬ 
tion  could  be  made.  Of  the  other  five  subjects,  two  gave  an  "indiffer¬ 
ence  band"  (i.e.,  chose  wager  (c)  for  x  -  800  and  x  -  850),  one  had  a 
"less  than"  choice  among  the  "at  least"  choices  (i.e.,  chose  the  "at 
least"  bet3  for  x  •  600,  650,  750  ,  800,  and  850,  but  the  "less  than" 
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* 

bet  for  x  -  700)  ,  and  the  other  two  subjects  chose  the  same  bet  all 

** 

nine  times  (one  subject  always  chose  the  "at  least"  bet  ,  while  the 
other  always  chose  the  (c),  or  indifference,  bet). 

Four  of  these  5  subjects  recognized  their  inconsistent  responses 
during  the  interview.  They  attributed  their  choices  to  carelessness 
and  all  four  indicated  a  desire  to  modify  their  choices.  The  subjeot 
who  always  chose  the  indifference  bet  did  not  seem  to  understand  the 
implications  of  his  choioe,  and  insisted  that  he  would  always  choose 
the  (c)  wager — even  if  the  values  used  in  the  experiment  were  replaced 
by  much  larger  values.  He  stated  that  he  had  chosen  the  indifference 
wager  "for  consistency"  and  refused  to  discuss  the  experiment  any  further. 

8.  CONCLUSIONS 

This  study  should  be  viewed  as  an  exploratory  one — an  initial 
attempt  to  probe  the  normative  implications  of  the  decision  theory 
postulates.  Several  characteristics  of  this  study  serve  to  distinguish 
it  from  others.  The  focus  was  on  the  level  of  the  individual  postulates 
rather  than  on  the  level  of  the  expected  utility  theorem.  The  various 
counter-arguments  to  the  postulates  were  examined  in  the  context  of 
controlled  experiments.  The  subjects  of  these  experiments  were  experi¬ 
enced,  practicing  decision  makers,  that  is,  high-level  business  executives. 


*  This  is  also  an  inadmissible  choice — a  violation  of  Postulate  3* ** 

**  This  choice  is  not  inconsistent  if  the  subject  thought  that  the  value 
would  be  greater  than  1000,  but  information  obtained  during  the  inter¬ 
view  indicated  that  this  was  not  so. 
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In  addition  to  gathering  evidence  on  the  extent  to  which  these  decision 
makers  conform  to  the  theory  in  their  actual  decision  behavior,  a  systematic 
followup  was  made  on  some  normative  implications  of  the  postulates.  In 
«he  process  some  preliminary  indications  of  the  teachability  of  the  theory 
were  obtained.  The  results  pertaining  to  each  of  these  points  will  be 
summarized  below. 

Not  surprisingly,  we  find  that  the  personal  probability /utility 
theory  is  not  a  very  good  description  of  the  decision  making  behavior 
of  the  subjects.  From  the  previous  sections  of  this  paper,  one  can  note 
a  high  degree  of  inconsistency  between  the  actual  choices  of  the  subjects 
and  those  choices  prescribed  by  the  appropriate  postulate.  In  addition, 
an  examination  of  the  responses  over  all  experiments  by  each  subject 
(see  Chapter  10  of  MacCrimmon  [19653)  shows  that  the  choices  of  each 
subject  were  inconsistent  with  at  least  one  of  the  postulates.  It 
follow*  logically  then  that  no  subject  acted  as  if  he  maximized  expected 
utility  over  all  experiments. 

In  attempting  to  generalize  this  conclusion,  two  characteristics 
of  the  experiments  must  be  considered.  First,  the  decision  problems 
were  specially  constructed  in  order  to  elicit  violations  of  particular 
postulates.  This  suggests  that  the  postulates  might  better  describe 
more  ordinary  situations — that  is,  the  less  structured  ones  we  would 
expect  to  confront  in  the  real  world.  On  the  other  hand,  the  decision 
problems  were  very  simple  and  somewhat  transparent  and  should  make 
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conformity  with  the  postulates  much  easier  than  in  the  more  complex 
real  world  decision  problems. 

The  results  presented  in  this  paper  show  that  there  was  a  high 

degree  of  apparent  violation  aoross  most  of  the  postulates  in  the  actual 

written  choices  of  the  subjects.  Later,  when  confronted  with  the  written 

prepared  answers,  they  tended  to  select  those  most  similar  to  their  own, 

although  there  was  a  slight  trend  toward  the  confroming  answer.  However, 

* 

during  the  interviews,  under  a  neutral  form  of  questioning  ,  there  were 
large  shifts  toward  conforming  with  the  postulates.  In  many  cases  the 
subjects  themselves  expressed  a  desire  to  change  their  choices  even 
before  the  particular  experiments  were  discussed.  The  violations  were 
generally  labelled  as  "mistakes"  by  the  subjects  and  in  one  or  two 
cases  may  be  attributed  to  undue  complexity  or  ambiguity  in  the  experi¬ 
ments  themselves.  Thus,  there  was  little  persistent  violation  of  the 
postulates.  Most  of  the  persistent  violation  that  did  occur  involved 
Postulate  2.  This  tends  to  justify  the  interest  in,  and  attention  to, 
Postulate  2  (Savage's  so-called  "sure-thing  principle")  in  the  literature. 

Most  of  the  persistent  violation,  and  also  the  highest  degree  of 
violation  in  the  actual  choices,  occurred  among  the  lower-level  execu¬ 
tives.  The  higher-level  executives  tended  to  have  both  a  higher  degree 


*  The  interview  results  show  that  the  subjects  did  not  know  the  context 
of  the  experiments.  It  had  been  repeatedly  emphasized  that  there  was  no 
right  or  wrong  answer  to  the  decision  problems  with  which  they  were  pre¬ 
sented.  For  details  on  the  exact  form  of  the  interview  see  MacCrimmon 
(1965 J. 
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of  conformity  in  their  actual  choices  and  &  better  acceptance  of  the 
applicability  of  the  postulate -based  reasoning.  The  higher-level 
executives  were  less  defensive  in  discussing  their  choices,  and  were 
able  to  draw  on  their  own  real  world  decision  making  experiences — a 
subject  that  was  pursued  in  the  latter  part  of  the  interviews.  In 
addition,  these  executives  (earning  over  $30,000/year)  recognized 
the  need  for  studies  of  decision  making  involving  uncertainty,  and  were, 
in  general,  enthusiastic  about  the  teachability  of  some  of  the  decision 
theory  concepts.  This  study  should  demonstrate  that  using  business 
executives  (especially  high-level  ones)  as  subjects  in  decision  making 
experiments  is  not  only  feasible  but  highly  desirable.  The  opportunities 
lost  by  using  college  students,  or  some  similar  group,  in  decision  making 
studies  should  be  recognized. 

Perhaps  the  most  important  conclusion  to  be  drawn  from  this  study 
i3  the  desirability  of  using  the  personal  probability /utility  postulates 
in  training  decision  makers.  As  noted  above,  the  most  successful  execu¬ 
tives  were  especially  enthusiastic  about  this  possibility.  Even  though 
training  was  not  an  overt  part  of  this  study,  that  is,  no  pressures  were 
applied  to  get  the  subjects  to  adopt  the  decision  theory  approach,  we  see 
that  simple  exposure,  as  described  in  this  paper,  led  to  a  large  shift 
toward  acceptance  of  the  postulates.  More  direct  training  methods  can 
easily  be  constructed — some  can  be  inferred  from  this  paper.  Future 
studies  should  systematically  examine  methods  for  training  people  how  to 
structure  decision  problems  and  how  to  police  their  own  decisions  within 
the  decision  theory  framework. 
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